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Abstract. We consider the arithmetic complexity of index sets of uniformly 
computably enumerable families learnable under different learning criteria. We 
determine the exact complexity of these sets for the standard notions of finite 
learning, learning in the limit, behaviorally correct learning and anomalous 
learning in the limit. In proving the Sg-completeness result for behaviorally 
correct learning we prove a result of independent interest; if a uniformly com- 
putably enumerable family is not learnable, then for any computable learner 
there is a enumeration witnessing failure. 



Algorithmic learning theory examines the process by which members of a class 
are identified from a finite amount of information. The classes to be learned are 
either classes of functions or classes of computably enumerable (c.e.) sets. In this 
paper we address the learning of c.e. sets. 

Since learning is not a mathematical concept, it is not endowed with an un- 
ambiguous definition. Like the concept of computability, learning has an intuitive 
meaning, but lends itself to a number of different formalizations. In learning theory, 
we consider effective formalizations and call them models of learning. A model is 
principally defined by two factors: the type of information and the criterion for 
success. The information is read by a learning machine from an enumeration of 
the set to be learned. As the machine must be computable, it cannot consider the 
entirety of an enumeration and will only take a finite initial segment as input. On 
such an input, the learning machine outputs a natural number, interpreted as a 
S^-code describing the content of the set being enumerated. We call such outputs 
hypotheses. As the machine reads longer initial segments of the enumeration, it 
outputs a sequence of hypotheses that we will call the hypothesis stream. The con- 
dition on when the hypothesis stream represents successful learning is the criterion 
for success and may depend on the accuracy, consistency or frequency of correct 
information in the hypothesis stream. Additional limitations, such as bounds on 
the computational resources of the learning machine, are often considered. 

The first model of learning is due to Gold [5]. According to his model, now 
commonly referred to as TxtEx-learning, a machine is deemed to have successfully 
identified an enumeration if, on cofinitely many initial segments, the machine out- 
puts the same hypothesis and it is correct. A machine is said to have learned a set if 
it identifies every enumeration of the set, and has a learned a family if it learns every 
member of the family. Three other standard notions are TxtFin-learning, TxtBC- 
learning and TxtEx*-learning which differ from TxtEx-learning in what constitutes 
successful identification of an enumeration. In the case of TxtFin-learning, while 
a machine is permitted to abstain from making a hypothesis for a finite amount 
of time, it must eventually output a hypothesis and the first such hypothesis must 
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be correct. A machine is said to have TxtBC-identified an enumeration if aU but 
finitely many of the hypotheses in the hypothesis stream are correct. In contrast 
to TxtEx-learning, they need not be the same. Last, TxtEx*-learning differs from 
TxtEx-learning in that the unique hypothesis appearing infinitely many times in 
the hypothesis stream need only code a set having finite symmetric difference with 
the content of the given enumeration. 

With this paper, we introduce a new line of inquiry to the field of learning theory. 
We examine the complexity of determining whether a family is learnable given a 
code for an effective presentation of the family, thereby establishing a measure of 
the complexity of the learning process. 

We prove that decision problems for learning under the standard notions of 
TxtFin-learning and TxtEx-learning are Sg-complete and I]4-complete, respec- 
tively, and that those for TxtBC-learning and TxtEx*-learning are both Eg-complete, 
when certain natural limitations are placed on the complexity of the families con- 
sidered. In proving the Eg-completeness of TxtBC-learning, we obtain a TxtBC- 
learning analog of a theorem of Blum and Blum . Blum and Blum's work demon- 
strated that a set is TxtEx-learnable if it is TxtEx-learnable from computable enu- 
merations. We show that, provided the family under consideration is uniformly 
computably enumerable (u.c.e.), one need only consider enumerations to decide 
if a family is TxtBC-learnable. 

We preface the completeness results with a brief introduction to some of the 
concepts of learning theory and notation from computability theory. For a more in 
depth treatment, we refer the reader to Osherson et al. [6\ and Soare [8]. 

1. Preliminaries 

Unless noted otherwise, all families in this paper are u.c.e. We regard a set. A, 
as coding a family, J^, where the i*'* member (or column) of T is {x : {i,x) £ A} 
for a computable pairing function {x,y). Given natural numbers, e and s, VFe,s 
denotes the result of computing the set coded by e up to s stages using a standard 
numbering of the c.e. sets. Finite strings of natural numbers are represented by 
lowercase Greek letters. Enumerations, called texts in learning theory, are either 
treated as infinite strings or as functions on the natural numbers. Initial segments 
of enumerations feature throughout this paper and are either denoted by lowercase 
Greek letters, as mentioned above, or by initial segments of functions: i.e. T[n] in 
learning theory notation, or / 1" n in standard logic notation. The n*'' element of an 
enumeration is denoted T(n) or /(n), as is appropriate. To switch from ordered lists 
to unordered sets, we say that content((T) = {a; G N : {3n){x = a{n))}. For infinite 
enumerations, we extend the content notation to denote the set that is enumerated. 
If A and B are sets of natural numbers and their symmetric difference, AAB, is 
finite, then we write A —* B. 

Learning machines are denoted by M or TV, with subscripts or superscripts 
as needed to indicate parameters. We consider an effective enumeration of all 
computable learning machines as having been fixed, whereby Af„ denotes the n*'' 
learner. 

Definition 1.1 (pj). Fix a symbol, '?', as a placeholder to indicate that a hypoth- 
esis has not yet been made. The definition of TxtFin-learning by a learner, M , is 
in four parts: 
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(1) M TxtFin-identifies an enumeration / if and only if (3n)(Vn' < n)(M{f\ 
7i') = ? A Mif\n) ^ ? A WMiftn) = content(/)). ' 

(2) M TxtFin-learns a c.e. set A if and only if M TxtFin-identifies every enu- 
meration for A. 

(3) M TxtFin-learns a family of c.e. sets if and only if Ad TxtFin-identifies 
every member of the family. 

(4) A family, J^, is TxtFin-learnable (denoted J- G TxtFin) if and only if there 
is a machine, M, that TxtFin-learns J-. 

Definition 1.2 ([5]). The definition of TxtEx- learning is analogous to that of 
TxtFin-learning. TxtFin is everywhere replaced by TxtEx and the first clause is 
replaced by: 

(1) AI TxtEx-identifies an enumeration / if and only if (limi_).oo M{ f\i) — 
n A Wn — content(/)). 

Definition 1.3 ([2])- The definition of TxtBC-learning is analogous to that of 
TxtFin-learning. TxtFin is everywhere replaced by TxtBC and the first clause is 
replaced by: 

(1) M Ta;tBC-i(ienii/ies an enumeration / if and only if (3n)(Vi > n){WM{f\i) — 
content(/)). 

Definition 1.4 ([7]). The definition of TxtEx*-learning is analogous to that of 
TxtFin-learning. TxtFin is everywhere replaced by TxtEx* and the first clause is 
replaced by: 

(1) A/I TxtEx* -identifies an enumeration / if and only if {3n) (Imii^ oo M{f\ 
i) — n A Wn —* content(/)). 

Before turning to our own results, we present the following facts which will be 
needed in the subsequent sections. 

We use the following theorem in the proof of Theorem l3.1l and the corollary that 
follows in the proof of Theorem 15.11 

Theorem 1.5 (Blum and Blum j4]). If a family is TxtEx-learned from computable 
enumerations by a computable machine M, then it is TxtEx-learned from arbitrary 
enumerations by a computable machine A/I . 

Corollary 1.6. If J- is TxtEx* -learned from computable enumerations by a com- 
putable machine AI , then it is TxtEx* -learned from arbitrary enumerations by a 
computable machine M . 

Proof. The proof is immediate. Simply replace TxtEx-learning with TxtEx*-learning 
throughout the proof of Theorem 11.51 □ 

Next we have Angluin's Theorem. The application of the theorem which follows 
is used in the proof of Theorem 13.21 

Theorem 1.7 (Angluin's Theorem [1]). Let C = {Lq, Li, . . .} be a uniformly com- 
putable family. C is TxtEx-learnable if and only if there is a u.c.e. family of finite 
sets J- = {Fq, Fi, . . .} such that 

(1) F, C Li for allieN 

(2) // Fi C Lj C Lj . then L, = L^ 
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Although we do not present it here, examination of the proof of Angiuin's Theo- 
rem shows that the existence of T does not require L to be uniformly computable, 
merely u.c.e. Example 11.81 uses this observation to distinguish TxtBC-learning and 
TxtEx- learning. 

Example 1.8. Let — {x + n : n < \Wx\}, and — {x + n : n ^ N}. Define 
F = {Hq, Lq, Hi, Li, . . .}. We claim that T is TxtBC-learnable, but not TxtEx- 
learnable. 

is clearly u.c.e. and can be enumerated so that is the (2e)*'' column and 
is the (2e + 1)'** column of -F. We must verify that is TxtBC-learnable, but that 
no machine can TxtEx- learn Consider a machine that, on input a, sets xq and 
xi to be the least element and greatest element, respectively, of content((j) and sets 
yi equal to the greatest element of W^.^ If yi > xi — xq, the machine outputs 
a code for Hxo', if xi — xq > yi, it outputs a code for L^g. Thus, for enumerations 
of infinite intervals, the machine may vacillate between two different correct codes. 
For finite intervals, eventually only one correct code will be output. We conclude 
that is TxtBC-learnable. 

Now, we wish to show that is not TxtEx-learnable. To obtain a contradiction, 
assume that we have a machine, AI, that TxtEx-learns J^. As we observed at the 
outset, this means there is a u.c.e. family {Go, Gi, . . .} such that each Gi is finite, 
G2i Q Hi, G2i+i Q Li and, ii Gi C A C B where B is the z*'* set in and A G J^, 
then A = B. Specifically, G2i+i C Li and G2i+i C Hi if and only if Hi = Li - 
exactly when Wi is an infinite set. Let m be the maximum number in G2i+i. If 
card{Wi) > m — i + l, then G2i+i Q Hi and Hi = Li. In other words, we can decide 
in the limit whether or not Wi is infinite. Since we cannot actually decide in the 
limit whether or not a number codes an infinite set, we have obtained the desired 
contradiction. 

We conclude this section with the following definition. 

Definition 1.9. Define the following four index sets of codes for u.c.e. families 
learnable according to the given criterion. 

(1) Define FINL to be the index set for TxtPin-lcarning. 

(2) Define EXL to be the index set for TxtEx-learning. 

(3) Define BCL to be the index set for TxtBC-learning. 

(4) Define EXL* to be the index set for TxtEx*-learning. 



We begin the presentation of our results by demonstrating that TxtFin-learning 
is Eg-complcte. This is accomplished in two steps. With Theorem l2.1l we place a S3 
upper bound on the complexity of FINL. Next, Theorem 12.21 reduces an arbitrary 
E!] predicate to FINL. 

Theorem 2.1. FINL has a E3 description. 

Proof. Suppose e codes a u.c.e. family C = {La,Li, . . .}. We will show that e S 
FINL if and only if 
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where (//(fc) denotes 

(Va, j)(3r -< a) ((Mfc(T) ^?) V (content(a) % L,) V (Mfc(a) =?) V (W^M,(a) = Lj)) . 

Observe that the formula mandates the existence of a learning machine, Mk, such 
that for every set in the family there is a string of elements from that set on which 
the learner outputs a hypothesis. Furthermore, if it outputs a least hypothesis, in 
the sense that the only hypothesis it makes on proper initial segments of the given 
data is ?, then that hypothesis is correct. We now build a new machine, M, based 
on Mfc, which TxtFin- learns the family jC. 

For a string a, define Aa = {t : content(T) C content((T) A |t| < |cr|}. Order Aa- 
by cr < T if either \a\ < \t\ or else \a\ = |r| and a is below r in the lexicographical 
order on nI"^'. Define M(ij) to be Mfc(T) where r is the least element of A^^ on 
which Mfe outputs a hypothesis other than ?. If no such t exists, M{a) =?. On 
no enumeration will the least hypothesis of M be incorrect since that would imply 
the existence of such an enumeration for M^. Fix an arbitrary enumeration, /, for 
Li € C Let CT be a string, with content(cr) C Li, on which Mk outputs a least (and 
hence correct) hypothesis. Every element of content((T) appears in /, thus there is 
an n such that a G Af\n- For some m < n, M{f\m) will be a least and correct 
hypothesis. 

Since the given formula is S3, we have produced a predicate with the desired 
properties. 

□ 

Theorem 2.2. FINL is T.'^-hard. 

Proof. Consider a S3 predicate P{e) O (3x)(Vj/)(Elz)(i?(e, x, z)), where i? is a 
computable predicate. We will reduce P to FINL by means of a computable function 
such that the image of e is a code for a u.c.e. family that is TxtFin-learnable if P(e) 
and not TxtFin-learnable if -'^'(e). We now fix e and proceed with the construction 
of a family based on that particular e. The family under construction is denoted 
Q = {Go, Gi, . . .}. While Q depends on e, we omit the parameter for the sake of 
simplicity as we are only concerned with the fixed value of e during the construction 
below. 

Each G € G will consist of ordered pairs and can thus be partitioned into columns 
C{G,i) — {x : G G}. For convenience, we will index the columns starting 

with —1. Let C{i) = {(«, .7;) : x e N}. For the remainder of the construction we will 
adhere to the notation defined in the following list. 

• Let {xs,ys, Zg) be a computable enumeration of all triples of natural num- 
bers. 

• For X > 1, let be the number of stages, up to s, at which the largest j, 
such that (Vy < j)(3i < s){xi = xAyi = y /\R{e,x — l, y, Zi)), has increased. 

• If it exists, let = lims_,.oo h^- 

• For X G N, an x-label is a number in the x*''-column, G(x), used to distin- 
guish sets in Q. Labels may be enumerated into any column of any G G G 
except G(G, —1) during the construction. We say G has an x-label k when 
k G G(G, x). Equivalently, when (x, k) G G. 

• Define = {G € G '■ G has an x-label k}. The family depends on the 
stage, but we do not include any notation to indicate the stage as it will be 
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clear from context. When we wish to reference the z^'-member of Sf^, we 
will write S^{i). 

• Let = card(<S^). 

• Define a function, , used to record numbers associated with each member 
of S^. In particular, p^{i) will be a number withheld from Sl^{i). Denote 
by P!^ the set {p^(0), . . . ,p^{n^)}. At each stage, we wiU ensure that \ 
{Pxii)} C C{S^ii),-l) and ^ C(5^(i),-1). At certain stages, the 
values of the will change. 

Next, we describe the actions taken at a given stage of the construction. The 
construction consists of using the predicate, P, to resolve two opposing forces. One 
is the attempt to label all sets in a unique way, and the other is to create an infinite 
family that mirrors the structure of {N \ {a;} : a; € N} every set of which has the 
same label. 

Stage s: The triple under consideration is {xs,ys,Zs)- Let t be the most recent 
previous stage at which xt = Xg. We examine two cases: > hi. and = h\. . 

First, suppose that h^.^ > h^^. We interpret this increase as progress toward 
verifying P{e). We enumerate elements as needed to ensure that, for each Xs- 
label, fc, and i < {x,,k) + h^^, if G,G' G 5^^, then C(G,i) n [0,(.T^,fc) + h%} = 
C{G',i) n [0, {Xs,k) + /i^J. If p^^(i) < {xs,kj+hl.^, we pick a member of C(-l) 
greater than (a;^, k) + h^^ and every number used in the construction so far, and 
set p^ (i) equal to the chosen number. We enumerate (i) into every member 
of S^^ \ {S^^{i)}. Thus, for each set with ccg-label k, there is a particular natural 
number the set does not contain, but which is contained in all other sets with 
ajg-label k. 

Finally, we pick the set of least index in Q that has not yet been assigned an 
a;s-label and assign it a unique, and previously unused, a;s-label. 

In the second case, suppose that — h^. . This stability suggests that the 
outcome will be ^P{e). For each k currently in use as an x^-label, we create 
a new set, S^^in^^ + 1) e <S^^, such that C{S^^{n^^ + D [0, {xs,k) + /i*J = 
C('5'x,("xJ,j)n[0,°(a;s,fc)+/iy, for j < k+h^^^. We enumerate P^*^ into 5^ 
and set (nj; +1) equal to the least member of C(— 1) not used during the 
construction so far and greater than {xs,k)+hl^. Finally, we enumerate p^^ {n^^ + 1 ) 
into every member of <S^^ \ {S^^{n^^ + 1)}. 

Verification: If P{e), then (3x)(Vy)(3z)(i?(e, x, y,z)). Hence, for some x, h% — >■ 
00. For infinitely many s, Xg = x, thus every set in Q will eventually receive an x- 
labcl. At such stages, agreement between sets with the same label is also increased. 
Consequently, any two sets in Q with the same x-label are equal. The family is 
learned by a machine that searches for the least a;-label and outputs a code for the 
first set in Q that receives the same x-label. 

If -P(e), then (Vx)(3i/)(Vz)(-i?(e, y, z)). Fix any machine, M. If M{a) = ? 
for every string a with content contained in a member oi Q, M has failed to learn 
Q and we are done. Otherwise, we may pick a string, a, such that 

• M{a) ^ ?; 

• for all T^a, M(r) = ?; 

• for some G gQ, content (ct) C G. 
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Let k be an x-label with which G is marked such that max(content(o')) < (xs , k) + 
h%^. Pick a stage, s, at which hy = hy for all y such that G contains a y-label less 
than or equal to (x, k) + h^. At a subsequent stage, t, Xt = x and a new set, G", is 
created containing content(cr). Labels contained in G' are either y-labels, fc, such 
that hy will never increase or labels enumerated into G" after stage t. The latter 
are only shared with sets created at subsequent stages. As a consequence, there is 
a member of never enumerated into G', but contained in G. We conclude that 
M , an arbitrarily chosen learning machine, has failed to TxtFin-learn Q. 

The computable function that maps e to a code for the u.c.e. family Q constructed 
above is a reduction of P to FINL. 

□ 

3. TxtEx-Learning 

We now proceed to describe the arithmetic complexity of TxtEx-learning. The 
first S4 description of EXL of which we are aware is due to Sanjay Jain. Here 
we present a different formula, but one which explicitly illustrates the underlying 
structure and serves as a model for the S5 description of BCL given in Section |4l 

Theorem 3.1. EXL has a E4 description. 

Proof. By Theorem 11.51 we need only consider computable enumerations when an- 
alyzing the complexity of EXL. Further, observe that if there is a machine, M, 
that TxtEx-learns a family, there is a total machine, M, that TxtEx-learns the 
same family. Specifically, define M{a) to be M{a\n) for the greatest n such that 
M{a\n) converges within |(t| computation stages and define M{a) = if no such 
initial segment exists. Suppose e codes a u.c.e. family {Lq, Li, . . .}. 

We will define a formula which states that there is a learner such that for every 
enumeration and every set in the family, if the enumeration is total and enumerates 
the set, then eventually the hypotheses stabilize and a given hypothesis is either 
correct or the hypotheses have not yet stabilized. Syntactically, this can be stated 
as follows: 

(1) {3a){yk,i)(j^Ma is total) A is total) A {(pk enumerates Li) ip{k,a,i)^ 

where we define ■(/'(fc, a, i) to be 

(3s)(V< > s)(Ma{(l)k \t) = Mai^k rs)) A iyn)(^WM^iMn) = L, 
V (3m > n){Ma{(t)k\m) ^ Ma(?!)fc fn))) . 

The last formula is A3. Thus, (H)) is S4 and characterizes TxtEx-learning because, 
for any family coded by a number e which satisfies formula ([T]), there is a learner 
whose hypotheses converge to correct hypotheses on every computable enumeration 
and if e fails to satisfy formula ([T]), every learner must fail on some hypothesis for 
some set in the family. 

We have, therefore, exhibited a E4 description of EXL. 

□ 
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To achieve the desired completeness resuh, we now prove that an arbitrary E4 
predicate can be reduced to EXL. The proof utihzes the family described in Ex- 
ample [LSI While not TxtEx-learnable, the family is learnable under more liberal 
descriptions of learning. It is, in a sense, just barely not TxtEx-learnable. 

Theorem 3.2. EXL is Yfl-hard. 

Proof. Let COINF be the index set of all codes for c.e. sets which are coinfinite. 
Since COINF is Ilg-complete, it suffices to prove that any predicate of the form 
(3x)(/(e, x) G COINF) for a computable function / can be reduced to EXL. As in 
Example II . 81 from the preliminary section, let He = {e+x : x < \We\}, Lg = {e + x : 
X G iu}, and T = {Hq, Lq, Hi, Li, . . .}. Fix a uniformly computable enumeration of 
T where H^, is the 2e -|- 1** column of J- and L^ is the 2e*'' column. For notational 
convenience, we denote the e*'* column of by Fg. We will define a sequence of 
u.c.e. families, TZn.e, and choose a computable map g so that g{e,x) is a code 
for the u.c.e. family Qe,x where, for x < e 

Qe^x — ^n,e- 
n(E [e,a:] 

We construct the u.c.e. families TZn,e simultaneously for x < n < e. The family, 
TZn,e, will consist of an infinite number of partial enumerations of Fn. How complete 
the enumerations are will depend on whether e e COF or e e COINF. 

Stage 0: Let TZn,e be the empty set. 

Stage s: Suppose that + j] C We,s- In this case, enumerate Fnj, a finite 
partial enumeration of Fn, and the least natural number greater than n/2 into the 
i*'' column of TZn,e- Denote this last number by no. We include no in order to 
guarantee that the set is nonempty and, if it is a partial enumeration of Ha or 
La, it will contain a. Also, for all k G [i,i + j], enumerate all the elements in the 
k*^ column into the z*'' column and vice versa so that all the columns with indices 
between i and i + j are identical. 

There are two cases. First, suppose e G COF, then there are only finitely many 
distinct sets in TZn,e] cofinitely many columns of TZn.e will be identical to Fn and the 
rest will be finite subsets of F„. Thus Qe,x will consist of F„ for a; < n < e together 
with some finite subsets of these sets. When e G COINF, TZn,e will contain only 
finite subsets of Fn and so Ge,x will consist of a collection of finite sets, possibly 
infinitely many. 

Based on g{e, x) and given an arbitrary E4 unary predicate P, we define a new 
map, h, which witnesses the reduction of P to EXL. Since P is E4, it is of the form 
{3y){Q{x, y)) where Q is a Ilg predicate. Let r be a one-to-one and computable map 
witnessing the reduction of Q to COINF. In other words, P{x) O {3y){r{x,y) G 
COINF). Furthermore, we may assume that 

P(e) -> (V°°y)(r(e,?/) G COINF) 

and 

-P(e)^(Vy)(r(e,2;)GC0F) 

Let s be such that for fixed e, {s{e,y)}y^ti is a computable, strictly increasing, 
subsequence of {r(e, y)}y^^. The existence of such a subsequence is guaranteed by 
the fact that r is one-to-one, implying that {r(e, ?/)}j,gN is an unbounded sequence. 
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For convenience, suppose that s(e, —1) = for all e G N. Let h he a. computable 
function such that, for e e N, h{e) is a code for the u.c.e. family 



If -'-P(e), then (^y){r{x,y) G COF). For each n G ui there is a y such that 
s{e,y — 1) < n < s{e,y). Fn e Qs(e,y+i),s(e,y)j hence C He- Recalling that is 
not TxtEx-learnable, we conclude that He is not TxtEx-lcarnable. 

If P(e), then {\/°°y){r{e,y) G COINF) and hence (V°°j/)(s(e, y) G COINF). Pick 
an no such that (Vn > no)(s(e,n) G COINF). For all n > no Gs{e,n+i).s(e.n) will 
consist entirely of finite sets. Furthermore, these sets will, by definition, contain no 
numbers less than no- On the other hand, every set in Gs{e,y+i),8{e,y) for y < uq 
will contain a number less than or equal to no or be finite. Therefore the whole 
family is learnable as follows. Let Mq be a computable function which learns the 
finite family Uj/<rao+i ^«(e.!/)>«(e>!/-i) ^"^^ ^® ^ computable function which 

learns the collection of all finite sets - in other words, a function which interprets 
the input it receives as a string and outputs a code for the content of that string. 
Define 



M{a) = \ "^^^^^ "° ^ content((T), 
1 Mi(cr) no ^ content (a). 

If M is fed an enumeration for a set in the family, then either no will eventually 
appear in the text or it will not. In either case, the learner will eventually settle on 
a correct code for the set. 

We have shown how to reduce an arbitrary predicate to EXL and we may 
conclude that EXL is S^-hard. 



4. TxtBC-Learning 

To prove the upper bound for BCL, we require a result of interest in its own 
right - independent of the arithmetic complexity of BCL. 

Theorem 4.1. Suppose that Q is a u.c.e. family. Either Q is TxtBC-learnable or, 
for each computable learner M, there is a enumeration of a set in Q that M 
fails to TxtBC-identify. 

Proof. Fix a u.c.e. family Q = {Go, Gi, . . . }. We must prove the following disjunc- 
tion. Either: 

(1) there is a computable machine which TxtBC-learns Q or 

(2) for any computable machine, M, either 



(a) there is a A" enumeration for a set G G Q on which M stabilizes to an 
incorrect answer or 

(b) there is a A2 enumeration for a set G G ^ on which M never stabilizes 

to codes for a single set. 



Assume that statement (2) is false. We may then fix a learner, M, that fails 
to satisfy statements (2) (a) and (2)(b). We shall demonstrate that, under this as- 
sumption, Q is TxtBC-learnable by some machine, i.e. statement (1) is true. To 
accomplish this, we perform a construction starting from a computable enumer- 
ation, g{0),g{l), . . ., oi G G Q uniformly obtained from an enumeration of the 




□ 
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family. The construction will follow a strategy designed to produce a enumera- 
tion witnessing statement (2)(b). Our assumption that these constructions fail will 
ultimately yield a method we shall use to build a learner for the family. 

We construct a enumeration, /, in stages. After stage s has completed, 
the state of the enumeration is a finite partial function, fs- Let ks{0), . . . ,ks{s) 
denote an increasing reordering of g{0), ■ ■ ■ , g{s)- In addition, we define a restraint 
function, rs(z), and a counter, is, which monitor the length of the enumeration 
and the initial segments of fs on which M exhibits key behavior. Specifically, is 
counts the number of times M appears to have output hypotheses coding distinct 
sets on fs, and rs{is) is the length of /«. For 1 < j < is, define the hypothesis 
hj = M{fs\rs{j)) and pick a least witness Xj G W/i|,sAW/(S_^,s. We will call hj 
and Xj the j*^ hypothesis and witness chosen at stage s, respectively. 

Stage s+1: Let fs, Ts, is, ks, Xq, . . . ,x\ and h^, . . . ,hf be as obtained from 
stage s. We shall refer to the preceding collectively as the variables. Let the finite 
sequence ks+i{0), . . . , ks+i{s + 1) be an increasing reordering of ^(0), . . ■,g{s + 1). 
Define a set of strings 

5(5 + 1) = {a: (|a|,y < s) A (content(a) C {ks+i{0), . . . ,ks+iis + 1)})}. 

We must consider two possible types of injury at the beginning of the stage. 

First, suppose that fcg+i \ + 1) 7^ ks\{s + 1). Let j < s he the least number 
such that ks+i{j) ^ ks{j). Reset the variables to their states at the beginning of 
stage j (for example, define fs+i to be fj). 

The second type of injury occurs when a witness is found either to be "wrong" 
or "not least". We call a witness, Xj, "wrong" if x| ^ Wh= ,s+i'^Wh'.__^^s+i and "not 
least" if the tuple fs \rs{j)) is not the least member of the set 

{{y,a) : (/Jr,(j -l)^a)A{ye W^h|,s+iAW^M(a),s+i) A (a £ 5(s + 1))}, 

where the set is ordered lexicographically and a <uex /3if|Q:|<|/?|or|a| = |/3| and 
a is lexicographically less than /3. Let j G N be least such that Xj is either "wrong" 
or "not least" and make the following changes to the variables. 

(1) is+i =j. 

(2) rs+i{m) = Vsim) for m < j and undefined for m > j. 

(3) fs+i \rs{j -1) = fs \rs{j - 1), and fs+i{x) is undefined for x > rs{j - 1). 

(4) Discard x^,. . . , xf^ and /i*, . . . , /if^. 

Having dealt with all required injury, we proceed to the actions of the stage. In 
particular, we search for the <ne2;-lcast pair in the set 

{{y,a) : {fs+i -< a) A {y e W^^j^ ,,+1 AM^M(a),s+i) A (a £ S{s + 1))}. 

If a least such pair, {y, a), is found, update the variables to refiect the successful 
search for an extension: 

(1) Increment is+i- 

(2) Extend fs+i to a. 

(3) Define rs+i{is+i) to equal the length of /s+i- 

(4) Update = a;| and = for j < is+i. 

(5) Define = y and hi^^-^ = M{a). 

On the other hand, if no such pair can be found, end the stage with no further 
changes. 
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Now suppose that lim5_>oo is = oo. Then, for any n, the n hypothesis and 
witness will be changed at most finitely many times. Therefore, lims^-oo x!^ and 
lim^^+oo fs \ n exist for every n, in which case our construction has produced an 
enumeration of G which is A2 and on which the hypothesis stream generated by 
M includes hypotheses that code different sets infinitely often. This is, of course, 
impossible since, by assumption, M TxtBC-learns Q from A2-enumerations. There- 
fore, lims^oo is 7^ 00. 

The construction was performed using a computable and uniformly obtained 
enumeration 17(0), 5(1), ... of G. The purpose of using a computable enumeration 
was to ensure that lims_>oo fs was A2. Because each pair of extension and witness 
are chosen in a canonical manner that is independent of the enumeration, any 
two instances of the construction will eventually select the same pair despite using 
different enumerations of G. This can be proved inductively. Suppose two different 
enumerations have produced two finite partial functions that agree on an initial, 
possibly empty, segment. Take the first point of disagreement. The choices of 
extension made at the point when the functions disagree cannot both be <iiex- 
least, therefore one will change at a subsequent stage. 

Define a computable function -0 such that ip{cr, s) = r, where r is the partial 
function that results from performing the construction on an initial segment, cr, 
of an enumeration after s stages of computation. Let Af (cr) — M{ip{a,\a\)). Fix 
an arbitrary enumeration q{0), q{l), ... of G. Since M TxtBC-learns Q from Aj- 
enumerations, there must be a longest partial function, a, that is cofinitely often 
extended by '0(g|"s,s). For any f3 such that content(/3) C G, we have VF^^^^^ — 

^M{a'fi) ~ Because G is an arbitrary member of G, M succeeds in TxtBC- 
learning Q. 

Since we have proved that Q is TxtBC-learnable assuming only that Q is TxtBC- 
learnable from Aj-enumerations, we have proved the desired claim. 

□ 

The above result allows us to place a bound on the complexity of the enumera- 
tions that must be considered when searching for an enumeration that witnesses a 
failure of TxtBC- learning. The next result applies Theorem 14. II to obtain an upper 
bound on the complexity of BCL - the first half of the completeness result for BCL. 

Theorem 4.2. BCL has a Eg description. 

Proof. Let M be an arbitrary learner and i an index for a set in the family 
J- = {i^Oi^ij • • •}• From a computable function, /, define a sequence of functions, 
{/sjsGN, by fs{x) = fis,x). Let (f>{M,f,i) be the formula 

(Vn, s)iWM{f, \n} = V {3n > n){3s' > s){WM{f^, \n') + M^Af(/, rn) 
K^s" >s'){f,.,W = fAn))). 

In words, (j){M,f,i) asserts that for any stage, s, and initial segment, fs\ri, 
either the hypothesis M{fs \n) is correct or there is a later stage and longer initial 
segment on which the A2-enumeration has stabilized and on which M outputs a 
code for a different set. 

Define tp{M, f ) to be the formula 

(3n)(Vn' > n){ys){WMU^\n) = M^m^ k) V (3s' > s){f,. \v! ^ f,\v!)) 
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and ^(/, i) to be 

(Vn)(3s)(Vt > - ft{n)) 

A(Vn,s)(3u,i > s){{fs{n) e V 7^ /^(n))) 

A(Va;, s){\/t > s){x e F,,u fs{n) = ft{n) A fs{n) = x). 

If ^j{M, /), then there is an initial segment of length n such that for any stage, s, 
and greater length, n', there are two possibilities. One, the hypotheses M outputs 
on fs\n and fs\n' code the same set. Two, there is a subsequent stage, s', at 
which the n' length initial segment changes: fs'\n' ^ fsln'. The formula £,{f,i) 
asserts that fs converges to an enumeration of Fi as s goes to infinity. In particular, 
lims_>oo fsin) exists for all n and lim^-^oo fsin) = a; if and only if x £ i^^. 

We must prove that the following is Sg and equivalent to e G BCL, where e 
codes a u.c.e. family {Fq, Fi, . . .}. 

(2) (3M)(V/,*)(^(/,*) ^^(M,/) A 0(Af, /,*)). 

Observe that ip{M,f) is E3. The formula 0(M, /, i) universally quantifies over 
the disjunction of a 112 formula and a formula. Thus, (/)(M, /, i) is n!]. Since 
^(/, «) is the conjunction of three n§ formulas, (,{f,i) is Ilg. From this, we conclude 
that 

e(/,i)^V(M,/)A<^(Af,/,i) 

is A!^. Consequently, © is Sg. 

To complete the proof, we must verify that any family that satisfies ([2]) is TxtBC- 
learnable. If ^(/, i), then / converges to a enumeration of F^. From ip{M,f), 
we have that there is an initial segment of the enumeration given by / such that, 
on longer initial segments of /, either the output hypotheses code the same set, or 
the A2 enumeration has not yet stabilized. Finally, (/'(M, /, i) states that for any 
initial segment either the hypothesis output by the learner is correct or the learner 
will output a later hypothesis that is different on an initial segment of / that has 
stabilized. 

Thus, if ([2]) is true, there is a computable learning machine M such that for any 
A2 enumeration /, M converges to consistent hypotheses on / and, if it has not 
yet output a correct hypothesis, it will change the content of its hypothesis at a 
later stage. This is clearly equivalent to TxtBC-learning T from Aj-enumerations. 
By Theorem 14. 1[ TxtBC-learning from Aj-enumerations is equivalent to TxtBC- 
learning from arbitrary enumerations for u.c.e. families. 

□ 

We present the lower bound in a modular fashion. The construction describes an 
attempt to diagonalize against every possible learner, which succeeds only if a given 
Eg predicate is false. A single step of the diagonalization is proved as a lemma. 

Lemma 4.3. Let M = Mm he a computable learning machine and We a c.e. set. 
There is a family Fm,e, uniformly computable in m and e, such that: 

(1) // We is coinfinite, then F„i,e is not TxtBC-learnable by M , but the family 
is TxtBC-learnable. 

(2) If We is cofinite, then Fm,e is uniformly TxtBC-learnable in both m and e. 

Proof. The construction will be performed in stages. During the stages, steps 
of a diagonalization process will be attempted, although these steps may not be 
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completed. The diagonalization is against the learner M and a step of the diag- 
onalization will be complete when a string is found on which the learner outputs, 
as a hypothesis, a code for a set that includes an element not in the content of the 
enumeration it has been fed. Such an element will be called a speculation. To be 
explicit, we define a natural number, x, to be a speculation of M on input a if for 
some s gN, X € WM{a),s and x ^ content((T). 

We will build a family Tm,e = {A, Bq, Bi, B2, ■ ■ ■}■ At each step i, the set 
Bi is initialized with the contents of the set A. A set, C, of speculations will 
be maintained. We reserve the 0*'' and 1** columns of each set for markers. If 
(0, j) S Bi, then B^ is said to have been tagged with j. Every set in the construction 
will contain (1,(0, m)) and (1,(1, e)) where m is a code for M. The rest of the 
construction occurs off the 0*'' and I'^'-columns and the 0*''-column of A is left 
empty. We now proceed with the construction of J^m,e- 

Fix M and We- 

Stage 0: C, A, Bq, Bi, . . . are all empty. Enumerate into A. Set uo = 0. 

Stage s: Suppose the first i steps have been completed. By C, A, Bq, . . . , Si+i 
we mean those sets in their current state. We are thus in the midst of step + 
Let wo,wi, . . . ,Wi enumerate the current members of C, where the index reflects the 
order in which they were chosen. Enumerate into each of the sets A,Bo,..., -Bj+i 
those Wj having j £ We,s- 

Next, we search for the least speculation, a: < s, of M on input as^a, for some a 
with |q:| < s, max(content(Q;)) < s, and content (a) n(C\^) = 0. If no spccidation is 
found, pick the least number neither in C\A nor the marker columns and enumerate 
this number into Bj+i, after which we end the current stage of the construction. If a 
speculation, x witnessed by a string a, is found, enumerate x into C and enumerate 
the members of {y : y ^ {C\A)Ay < max(content(a))} into A. Enumerate (0,i + l) 
into Bj+i. Prom this point on, only We is allowed to enumerate anything further 
into -Bi+i. Step z + 2 is now initiated by enumerating every element of A into 
Finally, we set ag+i = (Js'^aP, where /3 is an increasing enumeration of 
{y ■ y ^ {C \ A) A y < max(content(a))}. This ends the current stage of the 
construction. 

Observe that C\A are the speculations that, at the current stage, have not been 

enumerated into A. 

For coinfinite Wg there are two possibilities. If infinitely many steps complete, 
there is a subsequence {r^jngN of {i7s}nGN such that Wm{ts) for each s e N. 
Since (7^ and at are compatible for all .s, f G N, the computable function f{n) = 
(Tn{n) enumerates A. Thus, we have an enumeration for a set in the family on which 
M fails to converge to the correct set. If only finitely many steps complete, then 
there is a string a, equal to cts for some ,s G N, that has no extension witnessing 
speculation by M. The content of a is contained in the last nonempty B^, and Bi 
will become a cofinite set. Since M engages in no speculation beyond a, M must 
only output codes for finite sets, thus on any enumeration of that begins with 
the string a, M fails to TxtBC-learn Bi. 

Depending on the outcome of the construction, but independent of Wg, we can 
define a learning machine A'o that succeeds in TxtBC-learning J-m.e- 

Case 1: Suppose infinitely many steps of the construction complete. Define A^o 
to be a learner that outputs a code for A on any input string unless the string 
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contains (0, «) for sonic «, in which case it outputs a code for Bi. Nq succeeds in 
TxtBC-learning J-m.e- 

Case 2: If the j*''-step is the last step initiated, define A^o to be a learner that, 
on input cr, simulates the construction for A and outputs a code for one of A, 
Bq, Bi, . . . , Bj. If content((T) C A and (0,i) ^ content(cr) for any i < j, NQ{a) 
codes A. If (0, j) G content(a-), Nf){a) is a code for B^. Otherwise, No{a) is a code 
for Bj and A^o has TxtBC-learned J-m.e- 

Next, we define a machine that can learn UeeCOF meN-^™-^- ^ ^ COF. To 
distinguish it from the completed set, let Ag denote a simulation of the construction 
of A at stage s. 



The Nm.e will be defined below. Each Nm.e need only TxtBC-learn the family 
resulting from the construction based on M and We- For « G N, let A*,B* and 
{A[jBi\ ({0} eN))* denote S?-codes for A, Bi and AuSj \ ({0} eN), respectively. 
These codes can be computably derived from m and e. We define A^^.e as follows: 



where k denotes the greatest index of a set that has been used in the simulated 
construction up to stage |cr|. 

To determine if N^n^e TxtBC-learns the family, we must consider four cases, de- 
pending on the outcome of the construction and which set, D G Tm.e-, is enumerated 

to iV^^e- 

Case 1: Suppose D has a tag on the 0*''-column; in other words, there exists 
i G N such that D = B^. If the construction completes / < oo steps, then i < I. 
Otherwise, D may be any of the B^. Once {0,i) has appeared in the enumeration, 
the learner will hypothesize B* and never change hypothesis. 

Case 2: Suppose D = Bj where j is the index of the final, but incomplete, 
step of the construction. Since no (0, i) will ever be enumerated into Bj , the first 
case of N„i,e will never be satisfied. Cofinitely, since A is a finite set and Bj is 
not, the second case will not be satisfied either. {A is finite because only finitely 
many steps of the construction complete.) Thus, cofinitely, the learner will output 
(A U Bfe \ (0 © N))*, where k is updated at each stage to reflect the most recent 
addition to the family during the construction. Eventually k will stabilize to j, 
after which time the learner's hypotheses will always be correct. 

Case 3: Suppose D = A, where only flnitely many steps complete. Since A is 
finite, for all but finitely many s, A^ = A and we may replace A^^^ with A in the 
second case of the definition of Nm,e- Since no tag will ever be enumerated into 
the 0*''-column of A, the first case will never be satisfied and eventually the second 
case will always be satisfied and Nm,e outputs A* cofinitely. 

Case 4^ Finally, suppose D = A, where infinitely many steps complete. Note 
that because the learner is receiving an arbitrary enumeration, there need not be 
any correlation between the enumeration given to the learner and the enumeration 
of the simulation Ag. It is quite possible that the second case will be true only 




if card(({l} 8 N) n content((T)) < 1, 

Nm,e{(^) if (1,(0, m)), (1,(1, e)) Gcontent(a). 




if {0,i) G content(c7), 

if (0,i) ^ content(0-) Acontent(cr) C A^^^ 



^(AUSfe\({0}©N))* otherwise. 
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infinitely often. The first case, however, is never satisfied. All that remains is 
to prove that eventually the third case only produces correct hypotheses. Since 
e G COF, we may choose s such that [s, oo) C Wg- The set C\A is finite and 
{C\A)nBi = 0. Thus, AuBi\{0®N) = Abv i>s. 

Thus N succeeds in TxtBC-learning the following, possibly non-u.c.e., family 

eeCOF.meN 

and thus can TxtBC-learn any subfamily. 

□ 

Theorem 4.4. BCL is T,^-hard 

Proof. We wish to reduce an arbitrary Sg predicate P(e) to BCL. For an arbi- 
trary S4 predicate Q(e), there is a predicate, R{e,x,y), such that the following 
representation can be made: 

Q(e) O (3a)(V6)(i?(e,a,6)) 

o (3(a,s))[((V6)(i?(e,a,6))) A ((Va' < a){3s' < s){-nR{e,a',s'))) 

A {{3a' < a)(Vs' < s){R{e,a' ,s')))] 

<H- (3!{a,.s))[((V6)(i?(e,a,6))) A ((Va' < a){3s' < s){-nR{e,a' ,s'))) 

A {{3a' < a)(Vs' < s){R{e,a' ,s')))]. 

Since the predicate 

((V6)(i?(e,a,6)))A((Va' < o)(3s' < s){^R{e,a' ,s')))A{{3a' < a)(Vs' < s){R{e,a' ,s'))) 
is Ilg, for a suitable computable function g, 

g(e) ^ (3!a;)(3(e,x) G COINF) 

and 

-Q(e) ^ (Va;)(<7(e,x) G COF). 

Applying the above to P{e), the arbitrary S5 predicate under consideration, we 
may define a computable function / such that 

P(e) {3x)[{yx' > x){yy){f{e,x',y) G COF) 

A (Va;' < x){3^^y){f{e,x',y) G COINF)] 

and 

-P(e) ^ (Va;)[(3!y)(/(e,:r,y) G COINF)]. 

We will now define a family Qg from e such that Ge will be learnable if and only 
ifP(e). Define 

x,yeN 

Case 1: Suppose -'P(e). Then for every x, there ]s b. y for which f{e,x,y) G 
COINF. From this wc conclude that for each computable learner, M coded by to, 
there is a y such that f{e,m,y) G COINF. Qe contains a subfamily, J^m,f{e,m,y)^ 
that M cannot TxtBC-learn. Thus, Qe is not TxtBC-learnable. 
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Case 2: Suppose P(e) and let xq be such that (Va; > xo){Vy){f{e,x,y) £ COF). 
Let ao, ai, . . . , flfc enumerate the numbers less than xo such that, for unique corre- 
sponding bQ,bi, . . . , bk, we have /(e, a^, bi) G COINF and let Ki be a computable 
machine that learns J-ai.f{e.a bi)- The existence of such a machine is guaranteed by 
Lemma [4.3l Using the machine N from the proof of Lemma [4.3[ define a computable 
machine M on input string cr by 

Mia) = /^*('^) ^^'^'^^ ^ content((T) for z < fc, 

[iV(o') otherwise. 

If an enumeration of a set in the subfamily J'ai,f{e,ai,bi) is fed to M, then even- 
tually a tag in the l**-column will appear identifying it as such. Cofinitely often, 
the appropriate Ki will be used to learn the enumeration. If the enumeration is for 
a set from J^x.f{e.x,y) with either x ^ ai or y ^ bi for any i < k, then N will be 
used. From Lemma 14.31 it is known that N is capable of TxtBC-learning J-xj{x,y) 
for any x provided that y G COF. 

We conclude that BCL is E^-hard. 

□ 

5. TxtEx*-learning 

Our final collection of results borrows from the BCL lower bound arguments as 
well as the EXL description, given in Section S] and Section |31 respectively. We 
begin with a description of EXL*. 

Theorem 5.1. EXL* has a description. 

Proof. Suppose e is a code for a u.c.e. family {Lq,Li, . . .}. By Corollarv 11.61 a 
family that is TxtEx*-learnable from computable enumerations is TxtEx*-learnable 
from arbitrary enumerations. Further, observe that if there is a machine, M, that 
TxtEx*-learns a family, there is a total machine, M, that TxtEx*-learns the same 
family. Specifically, define M(cr) to be M{a \ n) for the greatest n such that Af (cr \ n) 
converges within \a\ computation stages and define M{a) = if no such initial 
segment exists. We proceeed with a formula nearly identical to the description of 
TxtEx- learning. Let Dq^ Di, . . . be a canonical, computable enumeration of the 
finite sets. Consider the formula 

(3) 

{3a){Vk,i){3i)(j^Ma is total) A (0^ is total) A {(f>k enumerates Li) — > ■il){k,a,i^l)^ 
where we define "0(^1 '^^j h ^) to be 

(3s)(Vi > s)(Ma{^k\t) = Ma{^k\s)) A {^n)(wMMu\n)^L, = D, 

V (3m > n){Ma{4>k \m) ^ Ma{4>k fr^))) ■ 

The only difference between the above formula and that of Theorem 13.11 is an 
additional existential quantifier over finite sets. Just as before, if a family satisfies 
formula ([3]) then there is a computable machine that identifies every computable 
enumeration for a set in the family. 

□ 

Lemma 5.2. Let M = Mm be a computable learning machine and We a c.e. set. 
There is a family Tm,e, uniformly computable in m and e, such that: 
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(1) If We is coinfinite, then J-„i,e is not TxtEx* -learnable by M , but the family 
is TxtEx* -learnable. 

(2) If We is cofinite, then J^rn,e is uniformly TxtEx* -learnable in both M and e. 

Proof Fix a machine M — Mm and a c.e. set We- We will construct a family, 
J^m,e — {A, Li, L2, i?2, • • •} in stages. Each set in Tm,e will have two columns 
({(0,x) : X e N} and : x G N}) reserved for markers. Every set in Tm,e 

contains (0, (m, 0)) and (0.(e,l)) and A contains the marker (0,(0.3)) as well. 
Unless otherwise indicated, any action during the construction is performed on the 
complement of the reserved columns. We identify this complement with N as it is a 
computable copy. At any stage of the construction, at most one pair of sets, Ln and 
Rn, will be actively involved in the construction. When there is such a pair, we call 
it the active pair and maintain an associated function, r„, which stores information 
about that pair. 

Stage 0: Search for the least string, cr, on which M outputs a hypothesis. Set cro 
equal to a and enumerate content (cto) into A. 

Stage s+1: Let -<...-< cjshe the sequence of strings passed to the current stage 
from stage s. If there is a currently active pair, L„ and _R„, then for j G We,s+i, we 
enumerate r„(j) and r„(j) + 1 into both I/„ and We then consider four cases 
depending on the status of two parameters. First, the existence of an active pair of 
sets. Second, the availability, within computational bounds, of an extension, a, of 
as on which M outputs a hypothesis different from its most recent hypothesis. We 
only consider the finite set of strings S = {(Js't ■ (|t| < s + 1) A (content (t) C s + 1)} 
in our search for a. 

Case 1: Suppose there is no active pair, but there is an extension, a (E 5, of Ug 
such that M(a) ^ M(crs). We pick the least such a. Set CTs+i = al3, where j3 is an 
increasing enumeration of {a; : a; < max(content(a))}, and enumerate content ((Tg+i) 
into A. 

Case 2: Next, consider the case where there is neither an active pair nor an 
a G S such that M{a) ^ M{as). Let n G N be least such that i„ and i?„ have 
not yet been used in the construction and set L„ and i?„ to be the active pair. Set 
ag+i = Og and enumerate content((7s+i) into both L„ and Rn- Pick the least even 
number, fc, such that k and k + 1 have not appeared in the construction so far. 
Enumerate k into _L„, fc + 1 into i?„ and set r„(0) = k. 

Case 3: Let L„ and i?„ be the active pair of sets, and suppose a G S' is least 
such that M{a) ^ M{(Js)- Set as+i = cfP, where P is an increasing enumeration 
of {x ■- X < max(content(a))}, and enumerate content (cts+i) into A- Next, we 
"cancel" the active pair. Specifically, we enumerate the marker elements (1, (n, 0)) 
and (1, (n, 1)) into L„ and i?„, respectively, and mark the pair as inactive. 

Case 4-' Finally, assume there is a currently active pair, L„ and R„, but no 
q; G S' such that M(a) M(tTs). Pick the least even number k larger than any 
number used in the construction so far, enumerate k into _L„, fc + 1 into i?„ and set 
r„(i + 1) = fc, where i is the greatest value for which r„(i) is defined. 

To verify that the above construction produces a family with the desired prop- 
erties, we must verify three statements: 

(1) J-m,e is TxtEx*-learnable for all M and e. 

(2) If We is coinfinite, then M does not TxtEx*-learn J-'m,e- 
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(3) If We is cofinite, then there is a machine, computable from m and e, that 
TxtEx*-learns J^m,e- 

If there is a pair of sets that remains active cofinitely, then J>„,e is a finite family. 
If no such pair exists, then every finite set has a unique marker by which it can be 
identified and the only infinite set is A. In either case, the family is learnable and 
we conclude that the first statement is true. 

To prove the second statement, we must again consider two cases. Suppose 
We is coinfinite. If a pair of sets remains active cofinitely, then M outputs the 
same hypothesis on all extensions of a finite partial enumeration whose content is 
contained in both members of the pair. Thus, there are two enumerations, one for 
each of L„ and on which M converges to the same hypothesis. The symmetric 
difference or _L„ and i?„, however, is infinite as one is co-odd and the other co- 
even. If no pair remains active infinitely, then there must be an infinite number 
of stages during the construction at which ctq -< ai -< ... are found such that 
M{(7s) 7^ M{(Js+i) and A is enumerated by /(n) = cr„(n). In either case, M fails 
to TxtEx*-learn Tm,e- 

Finally, we must exhibit a machine that can TxtEx* -learn all possible families 
Fm.e where e <E COF. In particular, a machine that can TxtEx*-learn the following 
possibly non-U. c.e. family as well as every subfamily: 

Q = [J J^m,e ■ 

eeCOF.meN 

Since We is cofinite for all the families under consideration, observe that Tm,e 
consists of a (possibly infinite) number of finite sets and either one or two sets {A or 
a pair L„ and that are cofinite in the complement of the marker columns. Fix 
codes ai and am,e such that Waa = 0, Wa-^ = N\{(a;, y) : {x = OVx = 1) Ay e N} 
and Wa^_^ is the set A G Tm,e- For notational ease, let C{k) = {{k,x) : x G N}. 
Define Nm,e by 

{ao if content(cr) n C(l) ^ 0, 
ara,e if (0, (0, 3)) € content(cr), 
a\ otherwise. 

Further, define a machine N by 

'7V„,e if (0, (m, 0)), (0, (e, 1)) e content(a), 
otherwise. 



N{a) 



To prove that N learns Q, select an arbitrary D G Q. Let e and m be the codes 
such that D G Tm,e for cofinite Wg. 

Case 1: Suppose that, during the construction of Tm,ei no pair of sets remains 
active infinitely. In this case, every member of Fm.e is marked, either with a marker 
in C(l) or with (0, (0,3)). Thus, N succeeds in TxtEx*-learning J^rn,e- 

Case 2: Suppose, on the other hand, a pair of sets remains active cofinitely 
during the construction. Let L„ and i?„ be that unique pair of sets. If D = ^, 
then (0, (0,3)) € D and cofinitely often Nm,e hypothesizes a^.e- Every other finite 
set contains a unique marker and is hence TxtEx*-learnable by Nm,e- Finally, if 
D = Ln OT Rn then D =* N\ (C(0) UC(1)). No initial segment of any enumeration 
of D contains either a marker in C(l) or the marker (0, (0, 3)). Thus, Nm,e again 
succeeds in TxtEx* -learning the set. 
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□ 

Theorem 5.3. EXL* is Ill-hard 

Proof. The proof is identical to that of Theorem 14.41 with one exception; in the 
conckision, we use Lemma 15.21 to justify the claim that N TxtEx*-learns J-xj{x,y) 
for any x provided that y G COF. 

□ 

6. Conclusion 

In summary, we have proved Sg-completeness for FINL, I]4-completeness for 
EXL and S^-completeness for both BCL and EXL*. Numerous other learning 
criteria are known to the theory, but their arithmetic complexities remain to be 
determined. 

One question stemming from the above work is to ask if there are natural classes 
other than u.c.e. families for which these complexity questions can be answered. 
Any candidate would have to provide a framework within which an upper bound 
could be placed on complexity. If more general classes are considered, observe that 
complexity can only increase, whereas for more restrictive classes complexity can 
only decrease. 
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